A Relationship between LDA and the Generalized Minimum Squared Error Solution

In this paper, a relationship between Linear Discriminant Analysis (LDA) and the generalized Minimum Squared Error (MSE) solution is presented. The generalized MSE solution is shown to be equivalent to applying a certain classification rule in the space defined by LDA. The relationship between the MSE solution and Fisher Discriminant Analysis (FDA) is extended to multi-class problems and also to undersampled problems for which the classical LDA is not applicable due to singularity of the scatter matrices. In addition, an efficient algorithm for LDA is proposed exploiting its relationship with the MSE procedure. Extensive experiments verify the theoretical results.